Computer-implemented system and method for integrating human observations into analytics data

ABSTRACT

A computer-implemented system and method for integrating user observations into operational data is provided. A database maintains notes each having received from a user and comprising a subjective observation. Operational data including workflow data of an objective nature is defined. Each of the notes is associated with one or more tags. The note associated with the tags is further maintained in the database. Criteria for retrieving the note are defined for the workflow data and forming a query for each of the workflow data. The query to select the note associated with the tags is executed for the workflow data based on the criteria and the selected note is integrated into the workflow data. The workflow data with the integrated note is displayed on a display.

FIELD

This application relates in general to data management and, inparticular, to a computer-implemented system and method for integratinguser observations into analytics data.

BACKGROUND

Organizations that carry out services in complex operational settings,such as governments, hospitals, banks, companies, and universities,process a tremendous amount of day-to-day transactions. Besides theirlarge scale of daily operations, organizations, especially organizationswith mobile workers or workers at multiple sites, such as localgovernments, including city transportation and public safetyorganizations, and hospitals, are required to administer variousunaccustomed events and circumstances on a daily basis. For example,local governments play roles in various functions, such as city or towndevelopment, tourism, public works, parks and recreation, police, fire,emergency services, transportation, housing, and so on. Similarly,services provided by hospitals vary greatly from patient to patient.However, current operating systems for organizations are not capable ofrecording and classifying all the business operations. For managing suchcomplex business operations, an operating system for organizations mustencompass a broad range of operations with due consideration to thechanging environment.

Commonly, organizational operating systems are divided into fourcategories of information systems, such as voice and text messagingsystems, workflow systems, data analytics, and structured documentcollection, and each system carries advantages and disadvantages. First,voice and text messaging systems carry information and coordinateactivities in organizations. Email, voicemail, and text messages areusually designed to deliver messages between individuals within anorganization by typically specifying a receiver of the messages. Thus,information regarding the messages are usually shared only between thesender and receivers. Even when the context of the message between thesender and the receiver shifts while exchanging messages in one messagethread, only the same individuals are involved in the message thread.Manually adding a new receiver into the message thread or specifying agroup of individuals as receivers can be an alternative to share themessage information with other individuals in the organization but thatis not a sufficient solution as an operating system. Further, themessage information cannot be processed as data and makes furtherprocessing, such as data analytics for aiding organizational activities,difficult.

Secondly, workflow systems orchestrate daily routine operations of theorganization into an accessible platform for use by individuals of theorganization. The workflow systems break organizational routineoperations into smaller tasks so that each individual in theorganization can efficiently process and manage a sequence of tasks.However, the workflow system is not adequate to respond to variable andcomplex environments as the workflow systems are designed for onlyfacilitating routine tasks. In other words, preparing detailedstep-by-step decision guidance for responding to complex environmentsand integrating human observations into the workflow system exceed acapacity of the workflow system.

Further, data analytics present a pattern in data by collecting andstatistically processing data. Data analytics can guide organizations intheir ongoing operations by reviewing and planning data, usually withvisualization. However, data analytics are quantitative and do notgenerally integrate open-ended, contextual, and unstructuredinformation.

Finally, document collection includes storing documents and metadata ina database and provides file repositories. The stored data in therepositories can be obtained by using a search function. However, forvariable types of documents and metadata, generality of the searchfunction is difficult. For example, calendars, spreadsheets, and eventplanning have special page types. Further, performing data analyticsamong variable types of data in the database has been unlikely to besuccessful. Thus, each current operating system falls short fororganizations to manage their complex operations.

Structured email systems are disclosed in Malone et al., “TheInformation Lens: Intelligent Information Sharing Systems,”Communications of the ACM, Vol. 30, No. 5, p. 390-402, May 1986 and Laiet al., “Object Lens: A ‘Spreadsheet’ for Cooperative Work,” ACMTransactions on Office Information Systems, Vol. 6, No. 4, p. 332-353,October 1986, the disclosures of which are incorporated by reference.Emails, such as only formulaic kinds of conversations, are structuredfor a computer system to access and process data elements.

Comments can be incorporated into analytics, such as described in Heeret al., “Voyagers and Voyeurs: Supporting Asynchronous CollaborativeInformation Visualization,” Proceedings of the ACM Conference on HumanFactors in Computing Systems (CHI), Apr. 28-May 3, 2007, San Jose,Calif. Sense.us, which is a system for collaborative visualization,provides a Web-based exploratory analysis framework for U.S. Censusdata. The Sense.us supports collaboration via commentary threadedconversations via views on data. The comments are connected to theanalytics data but do not become a part of data. Similarly, GoogleAnalytics, provided by Google, Inc., Mountain View, Calif., enable toattach comments by users on a visualized analytics data; however, thecomments are kept separate from the analytics data.

There is a need for organizing variable unstructured data andincorporating into the analytics data for managing and developingongoing organizational operations.

SUMMARY

A reflective analytics system which collects data outside of a regularoperational framework of an organization, such as individualobservations and information as notes or annotations that can be readand processed by both people and computers, and can integrate into theanalytics data generates system level knowledge and facilitatesoperational decision making with consideration of all the activities inthe organization including objective and subjective data. The systemadds tags to the notes to guide processing and can auto-fill variousinformation such as time and place of the note and other contextualinformation about the user's current activity. The system furtherincludes an intelligent processor that can analyze and retrieveinformation from the notes, displaying the notes in context inanalytics, and computing trends and other aggregated conclusions fromthe stream of notes.

One embodiment provides a computer-implemented method for integratinguser observations into operational data. A database maintains notes eachhaving received from a user and comprising a subjective observation.Operational data including workflow data of an objective nature isdefined. Each of the notes is associated with one or more of tags. Thenote associated with the tags is further maintained in the database.Criteria for retrieving the note are defined for the workflow data andforming a query for each of the workflow data. The query to select thenotes associated with the tags is executed for the workflow andanalytics data based on the criteria so that the selected note can beintegrated into the workflow data. The workflow data with the integratednote is displayed on a display.

A further embodiment provides a computer-implemented method forintegrating user observations into analytics data. Organizationalworkflow data of an objective nature including structured data foroperating transactions in an organization is maintained. A databasemaintains annotations and each annotation is received from a user whobelongs to the organization as a creator and comprising a subjectiveobservation. Each of the annotations is tagged with one or more of tags.The tagged annotation is embedded into the structured data bydetermining relevance of the tagged annotation to each of the structureddata. Data processing of the structured data is performed analytics databased on the processed structured data with the embedded taggedannotation is visualized on a user interface.

Still other embodiments of the present invention will become readilyapparent to those skilled in the art from the following detaileddescription, wherein is described embodiments of the invention by way ofillustrating the best mode contemplated for carrying out the invention.As will be realized, the invention is capable of other and differentembodiments and its several details are capable of modifications invarious obvious respects, all without departing from the spirit and thescope of the present invention. Accordingly, the drawings and detaileddescription are to be regarded as illustrative in nature and not asrestrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram showing a computer-implementedsystem for integrating user observations into analytics data inaccordance with one embodiment.

FIG. 2 is a flow diagram showing a computer-implemented method forintegrating user observations into analytics data in accordance with oneembodiment.

FIG. 3 is a process flow diagram showing, by way of example, anorganizational workflow for use in the method of FIG. 2.

FIG. 4 is a functional block diagram showing examples of notes for usein the method of FIG. 2.

FIG. 5 is a diagram showing, by way of example, a mobile interface forcreating a note.

FIG. 6 is a flow diagram showing a routine for associating tags withnotes for use in the method of FIG. 2.

FIG. 7 is a functional block diagram showing, by way of example, anontology of tags for use in the method of FIG. 2.

FIG. 8 is a flow diagram showing a routine for retrieving notes intoworkflow data for use in the method of FIG. 2.

FIG. 9 is a screenshot showing, by way of example, a Web page of a tableanalytic for reviewing each agent's shift in a team.

FIG. 10 is a screenshot showing, by way of example, a Web page of a mapand timeline analytic for reviewing overall performance of each agent.

FIG. 11 is a screenshot showing, by way of example, a Web page of atable analytic for reviewing performance of each agent for a specifictask.

FIG. 12 is a screenshot showing, by way of example, a Web page of atable analytic for planning citation goals.

FIG. 13 is a screenshot showing, by way of example, a Web pagedisplaying notes for use in the table analytic of FIG. 12.

FIG. 14 is a screenshot showing, by way of example, a Web pagedisplaying trends.

FIG. 15 is a screenshot showing, by way of example, a Web page of a mapand timeline analytic for displaying trends of accident and doubleparking with a note.

FIG. 16 is a screenshot showing, by way of example, a Web page of a mapanalytic for displaying trends of accident and double parking with anote responding to the note of FIG. 15.

FIG. 17 is a screenshot showing, by way of example, a Web page of a mapanalytic for displaying trends of accident and double parking with anote responding to the note of FIG. 16.

FIG. 18 is a screenshot showing, by way of example, a Web page of a mapanalytic for reviewing trends of accident and double parking.

FIG. 19 is a screenshot showing, by way of example, a Web page forsearching notes from a database.

FIG. 20 is a flow diagram showing a method of creating super tags.

DETAILED DESCRIPTION

Obtaining analytics data reflected with user observations assist anorganization to manage their ongoing daily operations more efficiently.FIG. 1 is a functional block diagram showing a computer-implementedsystem 10 for integrating user observations into analytics data inaccordance with one embodiment. Organization 11, such as governments andhospitals, employs many workers for various roles in the organization.Some of the workers are working on multiple sites for performing theirroles and mobile. Those mobile workers 12-15 can remotely access anorganization server 16 over a wide area public data communicationsnetwork 17, such as the Internet, using wired or wireless connectionsvia a desktop 12, portable 13, or mobile 14, 15 computers.

The organization server 16 manages a large scale of organizationoperating data (not shown) necessary for operating all the transactionsand other organization specific matters, as further described infra withreference to FIG. 3. The mobile workers 12-15 can access workflow data18 stored in a database 19 interconnected to the organization server 16through a workflow data interface 20 while they perform their tasks. Theworkflow data 18 can be provided to computers of the mobile workers12-15 through a Web interface 21, 22 or mobile application 23, 24, suchas a dashboard application, as well as other types of user interfaces.The workflow data 18 is used to manage routine and regular tasksconducted in the organization on a daily basis and mostly containobjective quantified data. For instance, the workflow data 18 regardingcity transportation systems can include parking violation data, ongoingconstruction data, and traffic congestion data. As another example, theworkflow data 18 regarding a public safety organization can includeaccident data and citation data. Other types of workflow data 18 arepossible.

While the mobile workers 12-15 are accessing the workflow data 18, theycan create a note or annotation 25 directed to the workflow data 18through the user interfaces 21-24 via the computers 12-15. The notes 25from the mobile workers 12-15 can include subjective observations andremarks regarding the workflow data 18, as further described infra withreference to FIG. 4. A note module 26 recognizes the notes 25 receivedfrom the mobile workers 12-15 through the workflow data interface 20 andstores the notes 25 in the database 19. Further, the notes 25 are taggedas they are created through the workflow data interface 20, as furtherdescribed infra with reference to FIG. 6. A tag module 27 canautomatically create tags 28 associated with the notes 25, such as timeand location data, an identifier of the mobile workers 12-15 who createthe notes 25, and work activity of the mobile workers 12-15. Further,the tag module 27 can provide predefined tags 28 to the mobile workers12-15 for a selection that best matches to the work activity performedby the mobile workers 12-15 or that best describes a nature or contextof the notes 25. In a further embodiment, the tags 28 can be manuallyentered by the mobile workers 12-15 to indicate a purpose of the notes25, a priority, and the length of the matter described in the notes 25.All the automatic tags, predefined tags, and manually entered tags aremaintained in the database 19. Other types of tags are possible.

The notes 25 associated with the tags 28 are maintained as tagged notes29 and integrated into the workflow data 18 based on criteria 30 of theworkflow data 18, as further described infra with reference to FIG. 8. Acriteria module 31 identifies criteria stored in the database 19 foreach workflow data 18 and selects a tagged note 29 which is relevant tothe workflow data 18. Then, an integration module 32 pairs the taggednote 29 with the relevant workflow data 18 and integrates the taggednote 29 as a part of the workflow data 18. A display module 33 sends theworkflow data 18 associated with the tagged note 29 for display on theuser interfaces 21-24. In a further embodiment, the tagged note 29 canbe aggregated into the workflow data 18 and processed for analytics. Bythe analytics process, the workflow data 18 and the tagged note 29 areanalyzed to obtain analytic data 35, such as statistical quantitativedata, by an analytic module 34. The analytic data 35 can be shown asmaps, timelines, charts, tables, and other structures on a display ofthe computers used by the mobile workers 12-15, as further describedinfra with reference to FIGS. 10-12.

Each computer 12-15 includes components conventionally found in generalpurpose programmable computing devices, such as essential processingunit, memory, input/output ports, network interfaces, and known-volatilestorage, although other components are possible. Additionally, thecomputers 12-15 and workflow server 18, analytics server 19, andintelligent engine 20 can each include one or more modules for carryingout the embodiments disclosed herein. The modules can be implemented asa computer program or procedure written as a source code in aconventional programming language and is presented for execution by thecentral processing unit as object or byte code or written asinter-credit source code in a conventional interpreted programminglanguage inter-credit credit by a language interpreter itself executedby the central processing unit as object, byte, or inter-credit code.Alternatively, the modules could also be implemented in hardware, eitheras intergraded circuitry or burned into read-only memory components. Thevarious implementation of the source code and object byte codes can beheld on a computer-readable storage medium, such as a floppy disk, harddrive, digital video disk (DVD), random access memory (RAM), read-onlymemory (ROM), and similar storage mediums. Other types of modules andmodule functions are possible, as well as other physical hardwarecomponents.

Integrating real-time user observations and intelligence into anorganizational operating system allows consideration of personalizedknowledge necessary for ongoing operations. FIG. 2 is a flow diagramshowing a computer-implemented method 40 for integrating userobservations into analytics data in accordance with one embodiment. Inan organization, routine tasks and transactions necessary to operate theorganization are often digitally managed by an organizational operatingsystem. The organization operating system can be a standard,enterprise-wide collection of business processes and run complexbusiness programs, such as workflow management system. The workflowmanagement system is a software system for monitoring a defined sequenceof tasks, arranged as a workflow. The workflow management system defineseach task for an individual or a group of individuals in theorganization and monitors ongoing processes regarding the task. Workflowdata can be defined for an individual in the organization who isaccessing the workflow data and displayed for the individual through auser interface, such as a Web interface application or mobileapplication, to perform the task (step 41).

A typical workflow management system is capable of controlling multiplelevels of tasks and individuals in the organization. By way of example,FIG. 3 is a process flow diagram showing an organizational workflow 50for use in the method of FIG. 2. In this example, three levels ofindividuals are involved in the organizational workflow, includingagents 51, supervisors 53, and managers 55. Agents 51 are operating 52individual tasks. Agents 51 typically report their performance of tasksthrough the organizational workflow to their supervisors 53 when theoperation is completed so that the supervisors 53 can review the tasks54. Managers and directors 55 oversee the organization and takeresponsibility on the activities in the organization as a whole.Managers 55 typically review 56 data which is statistically processed tosee any trend or course of actions requiring immediate attentions by themanagers 55. Then, the managers 55 can plan 57 goals to achieve certainoutcomes beneficial for the organization. The supervisors 53 areinformed about the plan from the managers 55 and create further plans 58for the agents 51 to perform. In this way, the organizational workflowcan be smoothly managed.

Referring back to FIG. 2, notes and annotations can be obtained fromusers of the operating system and maintained in a databaseinterconnected to the operating system (step 42). Notes and annotationscan be used by individuals in the organization for providing commentsand additional remarks regarding each task they are performing. Thenotes can be entered into the operating system when they review andupdate workflow data through a user interface, such as a Web interfaceapplication or mobile application. FIG. 4 is a functional block diagramshowing examples of notes 60 for use in the method of FIG. 2. The notes60 can be provided in various forms, such as text 61, pictures 62,animation 63, videos 64, and sound recordings 65. An individual can usetext 61 to describe the purpose and details of the note 60 and also canupload relevant pictures 62, animation 63, videos 64, and soundrecordings 65. When the user of the operating system creates a note viathe user interface, one or more tags can be associated with the note(step 43). Tags can be associated with each note in various ways, asfurther described infra with reference to FIG. 6. Tags are generallyclassified into categories. Thus, by tagging, the notes can becategorized based on a classification of the tags (step 44). Theclassification of the tags are further described infra with reference toFIG. 7. Each workflow data can specify criteria for retrieving notes.Notes for each workflow data can be selected by matching criteria withtags, and the selected notes are integrated into the workflow data (step45). Once integration of the notes into the workflow data occurs, thenote can be displayed in a context of the workflow data. Further, thenotes integrated into the workflow data can be processed for obtaininganalytics (step 46). In this way, the notes are directly integrated intothe operating system and organized in a layered structure. Further, theoperating system enables to reflect the user observations and remarksinto the system. In addition, annotations by operators in theorganization can provide further insights of actual operations in theorganization and enable the organization to properly interpret, review,and plan the organizational operations.

Notes and annotations can be best used by operators in organizations,such as city transportation systems, public safety organizations, andhospitals, whose activities often exceed regular and routine operationsdue to fast changing environment. An example scenario can help toillustrate.

Example Scenario 1 Time Activity Monday The traffic light turns yellowat N.E. 5^(th) and N.E. Howard. 7:30 AM Driver Smith quickly applies histruck's brakes and stops. Driver Jones is following close behind Smithand accelerating as he expected the truck to drive through theintersection. When Jones sees the flash of Smith's brake lights, his carcollided with Smith's truck. The back end of Smith's truck is slightlydamaged and the trailer hitch breaks through the grill and radiator ofJones' car. Steam erupts. Both vehicles stop. 7:35 AM The driversexchange information and Jones calls 911 for help. The Bay City PoliceDepartment responds to 911 call as the traffic accident takes placewithin the city limit. The Bay City Police Department has establishedprocedures for working with the citizens on the scene and also forcoordinating with emergency services, the highway patrol, and theDepartment of Motor Vehicles (DMV). The operator redirects the call to adispatcher for traffic operations. After receiving a radio call from hisdispatcher, Officer Jackson heads for the scene of the accident. 7:45 AMRealizing that his car is disabled, Jones also calls his insurancecompany. They take down some information and arrange for Terry's Towingto come for Jones' car. Terry's Towing dispatches a tow truck from S.W.Sandy Blvd. They tell Jones that a truck will be there in about 45minutes since the roads are busy during the morning commute. 7:50 AMOfficer Jackson arrives on the accident scene. He determines that nobodyis hurt. He tells the drivers to pull to the side of the road. Smithpulls over, but Jones' car is inoperative. 7:55 AM Officer Jacksonoffers to call a tow truck from Speedy Towing, which has a contract withthe city and a guaranteed response time of 15 minutes. Jones tellsOfficer Jackson that his insurance company has already arranged for atruck. 8:00 AM Officer Jackson advises both drivers to file reports withthe DMV within three days. Using his mobile device, he updates his dutystatus and enters in some of the information about the accident,including photographs from the scene. He puts out some traffic cones andflares and begins directing traffic around Jones' car, which is blockinga lane. 8:30 AM The tow truck from Terry's Towing arrives on theaccident scene. A driver of the tow truck makes his arrangements withSmith and the car is towed away. 8:35 AM Officer Jackson gets back inhis patrol car. From his mobile device, he closes out his activity onthe accident case. Because he had to direct traffic for 45 minutes whilewaiting for the tow truck from Terry's Towing to arrive, he adds a noteto the event. He clicks on the reason “waiting for tow” and types inthat Smith had arranged for towing with his own insurance company.

According to this scenario, Officer Jackson, an agent of the Bay CityPolice Department documents the delayed towing event by leaving a notefor explanation of the delay. The note can convey various forms ofinformation besides text messages, as further described supra withreference to FIG. 4. By way of example, FIG. 5 is a diagram showing, byway of example, a mobile interface 70 for creating a note. A note 71 iscreated by Officer Jackson in relation to his duty report regarding thecar accident illustrated in the above-discussed scenario. The note 71can include a title 72 to describe a purpose of the note 71. In oneembodiment, the note can be directed to a specific person 73, such as asupervisor or manager who reviews his duty report, or even anotherofficer. Other kinds of persons to direct notes are possible. In thisexample, the note 71 is directed to Supervisor Swanson #47. The note 71contains a field 74 for a user to enter in written information forproviding further details of the note 71. Further, the individual canupload pictures 75 on the note 71 and also indicates time 76 of thepictures 74 taken. Officer Jackson took two pictures 75 at each 7:55 amand 8:25 am at the car accident scene to provide further details to thenote 71. Further, the note can include a location 77 of the activityperformed by the operator. Finally, a tag 78 can be specified in thenote 71. FIG. 6 is a flow diagram showing a routine 80 for associatingtags with notes for use in the method of FIG. 2. Once all the necessaryinformation is entered in the note (step 81), a tag is specified. Tagscan be automatically entered along the note is created (step 82). Suchautomatic tags can include author's identity, current activity,location, and time. Other types of automatic tags are possible. Further,by a creator of the note, tags can be selected from a collection ofstandardized tags (step 83). Referring back to FIG. 5, predefined tagscan be provided by a pull down selection 79 and one or more tags 78 canbe selected for a single note. The predefined tags can includeaccidents, power outage, road repair, construction site, excessivehandicap stickers, and so on. In a further embodiment, the creator ofthe note can manually specify a tag in addition to or in lieu of thepredefined tags (step 84). In this way, the note can be associated withone or more tags (step 85) and help to identify and track the notesbased on the associated tags. Tags can specify time of relevance,priority, a specific group in relation to events, or even individualwhich the note can be delivered to. Other types of tags are possible.

Ontologically categorized tags can simplify maintenance of tags in theoperating system. Ontologies can be organized as directed graphs,hierarchies, list of lists as well as other forms of ontologies. FIG. 7is a functional block diagram showing, by way of example, an ontology oftags 90 for use in the method of FIG. 2. By way of example, a hierarchyof tags related to tasks of Transportation Division in the Bay CityPolice Department is shown. The tag hierarchy 90 shows parent categorytags 91-95 and child category tags 96-108. Child category tags 96-108further classify each parent category tag 91-95. In one embodiment, thechild category tags can be further categorized into grandchildcategories as well as further descendants. Further, new tags can beeasily created based on the ontology over time as new kinds of eventsand observations emerge as further described infra with reference toFIG. 20.

Notes associated with tags can be catalogued based on a classificationof the tags. For relating the tagged notes with a part of the workflowdata, each workflow data maintains criteria for retrieving notes. FIG. 8is a flow diagram showing a routine 110 for retrieving notes intoworkflow data for use in the method of FIG. 2. Each workflow datadefines criteria for retrieving notes (step 111). The criteria caninclude specific predefined tags maintained on the operating system,such as child category tags. Alternatively, the criteria can onlyindicate a parent category tags in the tag classification. Further, thecriteria can specify a specific location within a region, a specifictime in a time range, a specific date in a data range, or a specificperson in the organization, as well as other types of criteria. Once thecriteria is defined, the defined criteria are formed into a query forselecting notes (step 112). The query can be matched with a list of tags(step 113). For instance, if the query contains 4 tags, such as “DelayTow,” “Team 4,” “Week 49,” and “Enforce,” a note created by OfficerJackson which associated with tags of “Delay Tow” and “Team 4” can beselected as well as other notes which tags include some of the tags inthe query. The query can be matched to categories of tags, such asparent category tags (step 114). For instance, a query may include“Extreme Weather” tag which contains subcategories of “Heavy Snow” and“Flooding” (shown as 94, 103, and 104 respectively in FIG. 7).Alternatively, the query can be matched with a content of the note,including text, images, sound recordings, as well as other contents ofthe note where they are relevant. Once the notes are selected for eachworkflow data, the note with the tags are integrated into the workflowdata (step 115). Notes can be integrated potentially at several placesin the workflow data.

The operating system recognizes each integrated note as a part of theworkflow data and categorizes the note based on the associated tags.Thus, when performing analytics of the workflow data, the integratednote can be first identified as data of the workflow data and thencounted, combined, filtered, routed, and displayed in the specificanalytics where the contexts of the note are relevant. An examplescenario can help to illustrate how the integrated note is utilized anddisplayed on the operating system workflow data for reviewing.

Example Scenario 2 Time Activity Monday Supervisor Swanson reviews theactivities of the officers on 5:30 PM her squad for the day. Analyticsbring up statistics for their activities and also a flag on gaps in theofficer's day or other things that fall outside of expectations. Theanalytic shows that Officer Jackson has a note requesting an exceptionapproval on a “delay tow.” Swanson opens the note to see the explanationabout the exception. She sees that the insurance company for driverJones called own towing service from S.E. Bay City. That explains whyJackson took so long on the event. She clicks to “Approve Exception.”Friday A few days later, Supervisor Swanson reviews the perfor- 4:30 PMmance stats for her squad for the week before sending them up to hermanager. The statistics break out the timing different kinds ofassignments. The statistics for responding to and handling trafficaccidents are all within the normal 20-minute range. In addition,exception cases are called out. She notices that there were 4 instancesthis week of “waiting for tow” exceptions, with most of them under 60minutes and one event over 60 minutes.

According to the scenario, the note created by Officer Jackson appearson a shift review and an analytic of Officer Jackson's activities. FIG.9 is a screenshot showing, by way of example, a Web page of a tableanalytic 120 for reviewing each agent's shift in a team.

By way of example, the Web page 120 shows an analytic for reviewingshifts 121 displayed for Supervisor Swanson 122. Structured data showseach officer's performance on citations 123 in team 4, beat averages124, notes 125, and approvals 126. Supervisor Swanson 122 can see twoapprovals 127, 128 on her interface. When she clicks one of theApprovals “Delay Tow” 127, an approval request 129 is displayed as anindividual window. Any approval request must be usually accompanied withnotes to provide reasons for exceptions. In this case, the approvalrequest 129 is associated with a note created by Officer Jackson at 8:35am. Supervisor Swanson can take actions either to approve or reject therequest by clicking “Approve” icon 130 or “Reject” icon 131 in theseparate window 129. Supervisor Swanson can further have an option tocreate a note to the request by clicking “Add Note” icon 132 to enterany further information about the request or note. By approving therequest, the note associated with the request is further classified andcounted for data of “An approved exception” or data for “Approved towtruck” and no longer associate with data of “Approval request.” Eitherway by being approved or not, data such as when and where the eventoccur, which officer is involved with the approval request can be keptand tracked.

In a further embodiment, analytics can be shown as maps, timelines,charts, and other structures. FIG. 10 is a screenshot showing, by way ofexample, a Web page of a map and timeline analytic 140 for reviewingoverall performance of each agent. The Web page 140 shows shiftactivities of Officer Jackson. Activities of Officer Jackson can be seenin a map view 141, timeline view 142, and summary view 145. In thetimeline view 142, a large amount of time is assigned to “TT (towtruck)” event 143 in his duty status. Knowing that tow truck eventsgenerally take about 20 minutes, Supervisor Swanson can click an icon ofTT 143 and see note explanation in a separate window 144 created byOfficer Jackson regarding this TT event. Thus, even if there is norequest made by Officer Jackson for exception approval, the note canstill be displayed for Supervisor Swanson, such as a large time slot inthe timeline.

Similarly, tables can provide further details of the analytics. FIG. 11is a screenshot showing, by way of example, a Web page of a tableanalytic 150 for reviewing performance of each agent for a specifictask. A table 151 displays a summary of Team 4's performances,especially focusing on towing events. In one column 152, SupervisorSwanson can see a team summary of towing activities and find that excepttwo tow truck events, twenty one tow truck events were completed withintwenty minutes. In the next column 153, Supervisor Swanson can see thatthere are four exceptional cases but only one event taking more thansixty minutes. A further next column 154 shows that there was one towtruck event exceeding sixty minutes in the previous week.

The reflective analytic system can not only provide analytics forreviewing but for planning with consideration of notes. Typically,analytics implement statistical processing of quantitative objectivedata to obtain a certain pattern or trend in the data which ismeaningful for individuals in the organization to review and plan. Byintegrating notes into the workflow data with help of tags, theintegrated notes which typically include subjective qualitative data canbe further utilized and displayed into the workflow data for planning.An example scenario will help to illustrate.

Example Scenario 3 The primary mission of the Mountain City Right of WayEnforcement Department is to ensure public safety. The mission includesenforcing the right of way and parking regulations in Mountain City. Themission also includes activities like directing traffic at schools atpick-up and drop-off times. A second goal is to generate revenue forMountain City, since the funds collected from traffic and parking finesprovide important support for the city's budget. As part of its planningcycle, a manager in the Department is accountable to the mayor and citycouncil about revenue expectations. Goals vary for each month accordingto expected conditions. For example, months vary in the number ofworking days and holidays. The number of citations will also varyaccording to weather - such as when a snow storm shuts down MountainCity. Big tourist events and sports events in the city can also effectcitations if they involve parades or other traffic duty. Still anotherfactor is when schools are open or closed. In addition, there can belabor issues, such as staff vacations, retirements, or hiring thateffect the organization's performance.In this example scenario, a manager, for instance, Supervisor Swanson,can send a request to the operating system to perform analytics andestablish targets for citations each month of an annual plan. FIG. 12 isa screenshot showing, by way of example, a Web page of a table analytic160 for planning citation goals. The Web page 160 shows a target ofcitations for 2013 161 and the progress of the citations of 2013 as327,836 citations issued until June 162. A column 163 indicates thenumbers of citations for each month during the previous year. The nextcolumn 164 is designated to show actual numbers of the citations and thegoal of citations for each month is shown in the further next column165. The goal of citations for each month is adjusted for seasonal andevent related considerations. However, the number of the citations tendto get behind from the goal each month and can be indicated in theprogress of the citations 162, as “−32,596 behind.” A column for notes168 show notes relevant to citations planning. Numbers 169 above thenote 170 can indicate categories of each note. For instance, in January,four notes are recorded and Supervisor Swanson can understand at glancethat two notes are related to Weather 171 and other two notes arerelated to Events 172. The categories can be color-coded as well asother coding. In this Web page, 32,596 citations are short of the goaland by clicking “Distribute” icon 166, the shortage of the citations canbe distributed to the remaining months in 2013 as 167 so that thecitation goals can be successfully achieved. The distribution can bemanaged based on citation performance of the last year 163. However, thedistribution can also be adjusted by Supervisor Swanson after review ofnotes in each month. Other ways to manage distribution are possible.

Notes in the planning interface can supply interpretations andexpectations of analytic data and guide the manager or director to planahead with consideration of both quantitative and qualitative data. FIG.13 is a screenshot showing, by way of example, a Web page 180 displayingnotes for use in the table analytic of FIG. 12. A manager or directorcan review notes for explanation of the shortage of citations byclicking each note (170 in FIG. 12). For instance, four notes in January2013 181, including two tagged as Weather and the other two tagged forEvents can be shown as 182-184. A summary of each note is provided as alist 182-184 and by clicking one of the list 183, further details aredisplayed 186. The note contains a creator of the note, Dave Davis, dateof the creation, title, comments, pictures and tags 187. Since the tags187 include “Weather” and “Citation Planning,” the note can beappropriately displayed in the citation planning interface (160 in FIG.12). In this way, subjective observation introduced by notes can supplyquantitative data for the manager or director to consider reasons forlower citations.

Analytics can also be performed to diagnose organizational performanceas a whole and to revise priorities of activities in the organizationwith consideration of both subjective and objective data. An examplescenario helps to illustrate.

Example Scenario 4 In the organization, each position performs eachdifferent role. Managers set organizational priorities and isresponsible for the whole city. Supervisors oversee teams of officers,assigning beats and balancing schedules. And officers work their ownbeats, balancing public service activities and enforcement priorities.Day, Time Person Activity Monday, Manager Mandy Monroe, the manager ofparking and 9 AM Mandy traffic enforcement, reviews a dashboard showingcurrent trends in the city. The trends are intelligently generated fromnodes created by different people such as parking officers, citizen,police etc. Manager Mandy notices a recent trend related todouble-parking violations and accidents showing on the left side of thedashboard. She clicks this trend and the system shows more detailedsummary information about this trend on the right side of the dashboard.Manager Mandy finds there are increasing double parking violations andaccidents reported in area 42 around 9 am to 10 am on Tuesday andThursday. The day, time and location information are retrieved from theassociated tags and content of the notes that generating this trend.Some original notes generating this trend are shown below the summaryinformation. She clicks this trend and the system shows more detailedsummary information about this trend on the right side of the dashboard.Manager Mandy finds there are increasing double parking violations andaccidents reported in area 42 around 9 am to 10 am on Tuesday andThursday. The day, time and location information are retrieved from theassociated tags and content of the notes that generating this trend.Some original notes generating this trend are shown below the summaryinformation. Manager To confirm the information generated from the Mandynotes, Manager Mandy reviews the data of accidents and double-parkingviolations in area 42 on Tuesday and Thursday. She notices there hasbeen a rise in accidents and enforcement of double-parking restrictionshas been going down during 9:30 am to 10:00 am. She sees that deliveryvehicles were indirectly involved in many of the accidents. In severalcases delivery vehicles were double-parked adjacent to the accident.Sometimes an impatient driver behind the delivery truck drove around itand hit a jay-walking pedestrian. Monday, Manager Manager Mandy decidesto add a note to set a 9:10 AM Mandy high priority for enforcing doubleparking restrictions at the accident times in area 42. The priority,day, time and location information are tagged in this note, and are usedby the system to route the note to the right people automatically.Tuesday, Supervisor Susan Swanson is a sergeant who supervises 7:00 AMSwanson traffic enforcement in the AM shift. She likes to reviewstaffing assignments and priorities before her officers arrive for themorning shift. On Tuesday morning, Supervisor Swanson sees a note for aspecial assignment request from Manager Mandy in the complaints andrequests inbox on her dashboard. Supervisor Swanson was alerted to viewthis note because the system identified her as the supervisor for theteam that enforces area 42. Manager Mandy doesn't necessarily need toknow which supervisor will see her note. The system does it for herautomatically. Supervisor Swanson reviews the note by drilling down tothe details (detail information with a map of location and time). Shesees that there are three loading zone areas to be enforced on Tuesdayand Thursday morning effecting three officers on her team. The requesthas a high priority. Tuesday, Supervisor Supervisor Swanson then looksat her team to 7:10 AM Swanson see who is working and on the beats forthe following week. She notices that Officer Oliver is working on beat 5where the special enforcement request was made. Supervisor Swanson addssome information to the note to assigns Officer Oliver the specialassignment of enforce loading zones. Tuesday Officer Officer Oliver isalerted to see the note before 8:15 AM Oliver his shift begins. The noteis routed to him because he is tagged in the note and he is assigned tobeat 5, which is another tag in the note. Officer Oliver reviews thespecial assignments and beats and indicates on the device that heaccepts the special assignments. Tuesday, Officer Officer Oliverobserves the blocks in area 42 10:00 AM Oliver for 30 minutes and citedsome delivery vehicles for double-parking violation. He notices it is abusy commercial area and one of the reasons causing double-parkingviolations is the current loading zone is too small. After completingthis special assignment, he adds a note indicating he has finished theassignment and there is a need for larger loading zone. Monday ManagerManager Mandy checks to see (1) how the (two Mandy organization isresponding to her priorities, weeks and (2) whether accidents aredecreasing in later), response to increased enforcement. She sees 10:00AM the number of citation for double-parking violation has increased andthe number of accidents in the same area has decreased. She also noticesthere is a need to increase the loading zone spaces from the note inputfrom Officer Oliver, and puts it as an issue for future evaluation.

According to the illustration of the example scenario, a trend in thecity is generated by the operating system from the workflow dataincluding input from parking officers, citizen, and police. FIG. 14 is ascreenshot showing, by way of example, a Web page 190 displaying trends.Trends 191 show three trends 192, including accident and double parkingviolation 193. By selecting the trend of accident and double parkingviolations 193, a detailed summary 194 is showed with notes 195. Eachnote 196 can contain comments from creators of the notes 196 andautomatic tags can specify a day, time, and location information. FIG.15 is a screenshot showing, by way of example, a Web page of a map andtimeline analytic 200 for displaying trends of accident and doubleparking with a note. The map interface indicates a timeline view 201 andmap view 202 and shows a trend of increased incidents 203, 204 in thetime line view 201 and in the map view 202. Each dot in the circle 204represents one accident and a number in a bubble 205 represent a numberof citations nearby areas. Manager Mandy can create a note 206 to set ahigh priority 207 for enforcing double parking restrictions at9:30-10:30 am 208 on Tuesdays and Thursdays 209. The priority, day,time, and location information are tagged 207-209 and can automaticallyroute the note to individuals in a group, such as a group to whichManager Mandy belongs, a group specified by a level of priority, and agroup of each tag specifies. Other types of automatic routing arepossible.

A specific individual to whom the note is routed can add furtherinformation and tags to the note. FIG. 16 is a screenshot showing, byway of example, a Web page of a map analytic 210 for displaying trendsof accident and double parking with a note responding to the note ofFIG. 15. The note 211 provides a field of comments to add by a personwho is reviewing the note 211 and new tags 213, 214. In this examplescenario, Supervisor Swanson reviews the note created by Manager Mandyand responds with the request of increase for double-parkingenforcement. Supervisor Swanson instructs Officer Oliver who is workingon Beat 5 by way of tags 213, 214 and comments 212. FIG. 17 is ascreenshot showing, by way of example, a Web page of a map analytic 220for displaying trends of accident and double parking with a noteresponding to the note of FIG. 16. Also in this example, Officer Oliveris prompted to perform a special assignment from Supervisor Swanson andadds a note 221 responding to the note thread including comments fromSupervisor Swanson and Manager Mandy for showing that his assignment hasbeen completed in a field 222. Office Oliver can also initiate a noteinferring a necessity of the more loading zones.

A creator of the initial note can track and review the note and analyticany time after she issues the note. FIG. 18 is a screenshot showing, byway of example, a Web page of a map analytic 230 for reviewing trends ofaccident and double parking. The timeline analytic 231 displays anincrease in numbers of citations and decrease in numbers of accidents232. Similarly, the map analytic 233 displays a decrease of accidents inthe area 234 by way of showing an increase of citations and decrease ofaccidents. This example scenario illustrates how the organization can beguided by a top level of individuals, such as managers, in theorganization and activities responding to the guidance taken bysupervisors and agents in the organizations can be tracked and displayedfor review by the managers.

By way of overview, integrating notes based on human observations intodata for analytics supports better informed decision making in theorganization regarding daily activities which are typically outside ofdatabases and impractical to be incorporated as data. The capability ofthis reflective analytics system would further development of enterprisesoftware for organizations and demonstrate a wide range ofapplicability, such as local governments as “Smart Cities,” hospitals as“Smart Hospitals” and so on. Other examples of environment to apply thereflective analytic system are possible. Specifically, in traffic andparking enforcement systems, the reflective analytic system would provea strong value in many situations to integrate human observations,including increasing numbers of suspicious or fake permits, bagged orbroken meters, faded, defaced, or damaged signs, road hazards, upcomingconstruction projects, changes in the neighborhood and businesses,feedback regarding the size of beat, observations regarding a degree ofdanger in areas, number of handicap stickers in use in areas, and pricechanges of off-street parking. Other types of human observations arepossible.

As notes integrated into the reflective analytics system are organizedand managed with aid of tags, the notes can be easily searched andaccessed. FIG. 19 is a screenshot showing, by way of example, a Web page240 for searching notes from a database. Any individual in theorganization can use a search interface 240 to retrieve a note stored inthe database. The search can be conducted by inputting text in a field241 and executing a search query including the text. The execution ofthe search query can be done by matching the text with tags orcategories of tags. Other ways of retrieving notes are possible. Inaddition to the basic search function, a result of the search 242 can befiltered by filters 243. The filters 243 can limit the search result 242based on time range of search, a position or level of the creator in theorganization, a time of shift, region, squad or group, or a limitationof search area, such as note body, note tags, note author, and notecommentator. Other types of filters are possible. The search result 242can indicate a summary of the note, time and date of creation, tagsassociated with the note, as well as other information.

Tags are generally managed by the reflective analytics system. Overtime, the reflective analytics system can review usage of each tag, addand delete tags, and modify a classification of tags mentioned supra inFIG. 7. FIG. 20 is a flow diagram 250 showing a method of creating supertags. Super tags or categories can be created overtime as new kinds ofevents or observations are found. Example scenarios illustrate creatingnew tags in the ontology.

Example Scenario 5 Weather events can sometime affect operations in thecity. The Bay City experienced a heavy snowfall and high temperatureslast year. The operating system generates “# Heavy Snow” and “# HighTemperatures” tags for use by officers in the Bay City PoliceDepartment. A few months into use of the tags, the Bay City experiencesextremely heavy rain and flooding. These events result in use of “#Heavy rain” and “# Flooding” tags by the officers for providingexplanations to unusual events and incidents caused by heavy rain orflooding. In the absence of an ontology of tags, those tags cannot berecognized in the same family of the categories with “# Heavy Snow” and“# High Temperatures” tags and a new tag “# Extreme Weather” cannot begenerated as a super category.

Example Scenario 6 Sport events often put demands on city trafficcontrol and officer deployment. An operating system was put in placeand, in the first few months, tags such as “# Broncos Game” and “#Colorado Rockies” were used. Over the months, other tags, such as “#Denver Nuggets,” “# Colorado Avalanche,” and “Colorado Rapids” tags areadded. A super category such as “# Sporting Event” simplifiesorganization of tags.Since each workflow data contains criteria for retrieving notes and thecriteria are not often updated, super tags or categories would help toassociate new tags and workflow data. For instance, a workflow data,such as parking citation performance for Officer Jackson, contains “#Heavy Snow” and “# High Temperatures” as note retrieval criteria, “#Flooding” and “# Heavy Rain” tags will not be associated with theworkflow data. If a super category for all four tags, such as “ExtremeWeather” is created, the workflow data may be associated with “#Flooding” and “# Heavy Rain” when matching the criteria with thecategory of the tags. For creating a super tag or category, first,necessity of creating a new tag is identified (step 251). Conditionsindicating the necessity can be automatically determined based on anumerical threshold. For instance, if four similar tags in one categoryare identified, the number of four tags can trigger the creation of asuper tag. Other ways to automatically identify the necessity arepossible. Alternatively, a super tag can be manually created by anyindividual in the organization or a group of individuals in theorganization who has authorization to make such addition. The new supertag can be created and the classification of tags are reclassified (step252). Other circumstances to create a super category or tag arepossible.

While the invention has been particularly shown and described asreferenced to the embodiments thereof, those skilled in the art willunderstand that the foregoing and other changes in form and detail maybe made therein without departing from the spirit and scope of theinvention.

What is claimed is:
 1. A computer-implemented method for integratinguser observations into operational data with the aid of a digitalcomputer, comprising the steps of: maintaining notes in a databasecomprised in a storage medium, each note having been received from auser and comprising a subjective observation; maintaining in the storagemedium a plurality of categories of tags, each of the categoriescomprising a plurality of the tags; defining operational data comprisedin the storage medium and comprising workflow data of an objectivenature; associating each of the notes with one or more of the tags witha computer processor and memory with the computer processor coupled tothe non-transitory computer readable storage medium and furthermaintaining the note associated with the tags in the database with thecomputer; defining criteria for retrieving one or more of the notes forthe workflow data, the criteria comprised in the storage medium andforming a query for each of the workflow datum based on the criteria,the query comprising at least one of the categories of the tags;executing the query with the computer to select one or more of the notesassociated with the at least one of the tags comprised in the at leastone category in the query for the workflow data and integrating theselected notes with the computer into the workflow data; and displayingthe workflow data with the computer with the integrated note on adisplay.
 2. A method according to claim 1, further comprising:generating with the computer summaries of each of the notes; andutilizing with the computer the summaries of the notes comprising atleast one of: analyzing with the computer a trend of the notes fororganizing the categories of the tags based on the summaries; andidentifying with the computer the workflow data matching with thesummaries of the notes.
 3. A method according to claim wherein theclassification of the tags is organized as at least one of a directedgraph, hierarchy, and lists of lists.
 4. A method according to claim 3,further comprising: recognizing with the computer one or more of thecategories which are related each other; and creating with the computera category above the related categories as a new category.
 5. A methodaccording to claim 1, further comprising: providing with the computerpredefined tags to each note comprising information at least one of day,time, and location when the note is created, identifier of the user, andwork activity of the user.
 6. A method according to claim 1, furthercomprising at least one of: providing with the computer predefined tagsfor selecting by the user for associating with the note; and providingwith the computer a field on a user interface to enter tags manually bythe user for associating with the note.
 7. A method according to claim1, wherein the notes comprise at least one of text, pictures, videos,animation, and sound recordings.
 8. A method according to claim 1,wherein the notes comprise at least one of time created, location, titleof the note, name of the user, comments, priority, request forexception, and purpose of the note.
 9. A method according to claim 1,further comprising: identifying with the computer a receiver of the notewhen the tag is associated with the note; and routing with the computerthe note associated with the tag to the identified receiver.
 10. Amethod for integrating user observations into analytics data with theaid of a digital computer, comprising the steps of: maintainingorganizational workflow data of an objective nature comprisingstructured data comprised in a storage medium for operating transactionsin an organization; maintaining in the storage medium a plurality ofcategories of tags, each of the categories comprising a plurality of thetags; maintaining annotations in a database comprised in a storagemedium, each annotation having received from a user who belongs to theorganization as a creator and comprising a subjective observation;tagging with a computer processor and memory with the computer processorcoupled to the non-transitory computer readable storage medium each ofthe annotations with one or more of the tags; embedding with thecomputer the tagged annotation into the structured data by determiningwith the computer relevance of the tagged annotation to each of thestructured datum, comprising defining criteria for retrieving the taggedannotation for the workflow data, the criteria comprised in the storagemedium, forming a query for each of the workflow datum based on thecriteria, the query comprising at least one of categories of the tags,and executing the query with the computer to select the taggedannotation; and performing data processing of the structured data withthe computer and visualizing, on a user interface with the computer,analytics data based on the processed structured data with the embeddedtagged annotation.
 11. A method according to claim 10, furthercomprising: displaying with the computer the analytics data with theembedded tagged annotation for one group of the users in theorganization on the user interface; receiving with the computer arequest for obtaining another analytics data from other group of usersin the organization; rearranging with the computer the analytics datafor use by the other group of users; and presenting the rearrangedanalytics data on the user interface with the computer for the othergroup of users with the embedded tagged annotation.
 12. A methodaccording to claim 10, further comprising: determining with the computerthe tagged annotation as relevant for the structured data by matchingeach query for each of the structured data with each category for eachtag.
 13. A method according to claim 10, further comprising: maintainingwith the computer an interface for search of the annotation in thedatabase, comprising: generating and storing with the computer summariesof the annotations in the database; performing with the computer searchof the annotations based on at least one of the summaries and tagsassociated with the annotations; and displaying with the computer aresult of search of the annotations.
 14. A method according to claim 13,further comprising: filtering with the computer the result of search fordisplay by at least one of time window, location, identity of the userwho created the annotation, and ranking of the user who created theannotation.
 15. A method according to claim 10, further comprising:identifying with the computer an alert attached to the embedded taggedannotation; determining with the computer one or more of receivers ofthe embedded tagged annotation based on the alert; displaying with thecomputer the embedded tagged annotation to the determined receivers onthe user interface.
 16. A method according to claim 15, furthercomprising at least one of: identifying with the computer a group of theusers of the structured data as the receivers based on at least one ofsummary of the annotation and the tag; and receiving, from at least oneof the users in the organization, with the computer one or more ofspecific users of the structured data as the receivers.
 17. A methodaccording to claim 15, further comprising: receiving with the computer aresponse to the alert by at least one of the receivers; and reflectingwith the computer the response to the embedded tagged annotation.
 18. Amethod according to claim 10, wherein the analytics data is shown as atleast one of maps, timelines, charts, and tables.
 19. A method accordingto claim 10, wherein the user interface is provided as at least one ofdashboard application and Web application.